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Recovery  of  Compressible  Signals 
in  Unions  of  Subspaces 

Marco  F.  Duarte,  Chinmay  Hegde,  Volkan  Cevher,  and  Richard  G.  Baraniuk 
Department  of  Electrical  and  Computer  Engineering 
Rice  University 


Abstract — Compressive  sensing  (CS)  is  an  alternative 
to  Shannon/Nyquist  sampling  for  acquisition  of  sparse  or 
compressible  signals;  instead  of  taking  periodic  samples, 
we  measure  inner  products  with  M  <  N  random  vec¬ 
tors  and  then  recover  the  signal  via  a  sparsity-seeking 
optimization  or  greedy  algorithm.  Initial  research  has 
shown  that  by  leveraging  stronger  signal  models  than 
standard  sparsity,  the  number  of  measurements  required 
for  recovery  of  an  structured  sparse  signal  can  be  much 
lower  than  that  of  standard  recovery.  In  this  paper,  we 
introduce  a  new  framework  for  structured  compressible 
signals  based  on  the  unions  of  subspaces  signal  model, 
along  with  a  new  sufficient  condition  for  their  recovery  that 
we  dub  the  restricted  amplification  property  (RAmP).  The 
RAmP  is  the  natural  counterpart  to  the  restricted  isometry 
property  (RIP)  of  conventional  CS.  Numerical  simulations 
demonstrate  the  validity  and  applicability  of  our  new 
framework  using  wavelet-tree  compressible  signals  as  an 
example. 

Index  Terms — Compressive  sensing,  compressible  sig¬ 
nals,  unions  of  subspaces 

I.  Introduction 

Compressive  sensing  (CS)  is  a  new  approach  to  simul¬ 
taneous  sensing  and  compression  that  enables  a  poten¬ 
tially  large  reduction  in  the  sampling  and  computation 
costs  at  a  sensor  for  signals  having  a  sparse  representa¬ 
tion  in  some  basis.  CS  builds  on  the  work  of  Candes, 
Romberg,  and  Tao  [1]  and  Donoho  [2],  who  showed 
that  a  signal  having  a  sparse  representation  in  one  basis 
can  be  recovered  from  a  small  set  of  projections  onto 
a  second,  measurement  basis  that  is  incoherent  with 
the  first.1  Random  projections  play  a  central  role  as  a 
universal  measurement  basis  in  the  sense  that  they  are 

Email:  {duarte,  chinmay,  volkan,  richb}@rice.edu;  Web: 
dsp.rice.edu/cs.  This  work  was  supported  by  the  grants  NSF 
CCF-0431150,  CCF-0728867,  CNS-0435425,  and  CNS-0520280, 
DARPA/ONR  N66001-08-1-2065,  ONR  N000 14-07- 1-0936, 
N000 14-08- 1  - 1067,  N00014-08-1-1112,  and  N00014-08-1-1066, 
AFOSR  FA95 50-07 -1-0301,  ARO  MURI  W31  INF-07-1-0185,  and 
the  Texas  Instruments  Feadership  University  Program. 

'Roughly  speaking,  incoherence  means  that  no  element  of  one 
basis  has  a  sparse  representation  in  terms  of  the  other  basis. 


incoherent  with  any  fixed  basis  with  high  probability. 
The  CS  measurement  process  is  nonadaptive,  and  the 
recovery  process  is  nonlinear,  for  which  a  variety  of 
algorithms  have  been  proposed  [1-6]. 

While  this  represents  significant  progress  from 
Nyquist-rate  sampling,  it  is  possible  to  do  even  better 
by  more  fully  leveraging  concepts  from  state-of-the- 
art  signal  compression  and  processing  algorithms.  In 
many  such  algorithms,  the  key  ingredient  is  a  more 
realistic  signal  model  that  goes  beyond  simple  sparsity 
by  codifying  the  inter-dependency  structure  among  the 
signal  coefficients.  Coding  the  coefficients  according  to 
a  model  for  this  structure  enables  these  algorithms  to 
compress  signals  close  to  the  maximum  amount  possi¬ 
ble  —  significantly  better  than  a  naive  coder  that  just 
processes  each  large  coefficient  independently. 

We  have  previously  developed  a  new  model-based 
CS  algorithmic  framework  [7]  that  parallels  the  conven¬ 
tional  theory  by  establishing  an  underlying  signal  model , 
favoring  certain  configurations  for  the  magnitudes  and 
indices  of  the  significant  coefficients  of  the  signal,  and 
designing  recovery  algorithms  that  exploit  the  knowledge 
of  such  a  model.  Building  on  this  work,  we  developed 
CS  recovery  algorithms  in  [8]  for  the  recently  introduced 
union-of-subspaces  models  for  strictly  sparse  signals  [9- 
11].  We  have  shown  that  these  algorithms  achieve  the 
same  performance  as  standard  CS  recovery  [8].  In  both 
cases,  by  reducing  the  degrees  of  freedom  of  a  sparse  or 
compressible  signal,  signal  models  provide  two  immedi¬ 
ate  benefits  to  CS.  First,  they  enable  a  reduction  in  the 
number  of  measurements  M  required  to  stably  recover  a 
signal.  Second,  during  signal  recovery,  they  enable  us  to 
better  differentiate  true  signal  information  from  recovery 
artifacts,  which  leads  to  a  more  robust  recovery. 

In  this  paper,  we  build  on  previous  work  by  ex¬ 
panding  the  union-of-subspaces  signal  models  beyond 
strictly  sparse  signals.  To  precisely  quantify  the  benefits 
of  model-based  CS,  we  introduce  and  study  several 
new  theoretical  concepts  that  could  be  of  more  general 
interest.  We  define  a  class  of  model-compressible  signals 
that  can  be  approximated  by  increasingly  complex  sig- 
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nals  under  a  established  union-of-subspaces  model  with 
exponentially  decaying  error.  We  also  formulate  a  new 
restricted  amplification  property  (RAmP)  that  guarantees 
stable  recovery  of  model-compressible  signals.  For  some 
compressible  signal  models,  the  number  of  random  mea¬ 
surements  M  required  by  the  RAmP  is  independent  of 
N. 

This  paper  is  organized  as  follows.  A  review  of  the 
CS  theory  is  given  in  Section  II.  Section  III  develops 
the  concept  of  model-sparse  signals  and  introduces  the 
concept  of  model-compressible  signals.  We  also  quan¬ 
tify  how  signal  models  improve  the  measurement  and 
recovery  process  by  exploiting  the  model-based  RIP  for 
model-sparse  signals  and  by  introducing  the  RAmP  for 
model-compressible  signals  in  Section  IV.  Section  V 
then  specializes  our  theory  to  the  case  of  wavelet  tree- 
sparse  signal  models  and  Section  VI  reports  on  a  series 
of  numerical  experiments  that  validate  our  theoretical 
claims.  We  conclude  with  a  discussion  in  Section  VII. 

II.  Background  on  Compressive  Sensing 

Given  a  basis  T,  we  can  represent  every  signal  x  6 
Rn  in  terms  of  the  coefficient  vector  a  as  x  =  'Pa.  In 
this  section  we  will  assume  without  loss  of  generality 
that  the  signal  x  is  sparse  or  compressible  in  the  canoni¬ 
cal  domain  so  that  the  sparsity  basis  T  is  the  identity  and 
a  =  x.  A  signal  x  is  K-sparse  if  only  K  <C  N  entries  of 
x  arc  nonzero.  We  call  the  set  of  indices  corresponding 
to  the  nonzero  entries  the  support  of  x  and  denote  it  by 
supp(x).  The  set  V k  of  all  K -sparse  signals  is  the  union 
of  the  (^),  if -dimensional  subspaces  aligned  with  the 
coordinate  axes  in  RN . 

While  many  natural  and  manmade  signals  are  not 
strictly  sparse,  they  can  be  approximated  as  such;  we 
say  that  such  signals  are  compressible.  An  example  is 
a  signal  x  whose  coefficients,  when  sorted  in  order  of 
decreasing  magnitude,  decay  according  to  the  power  law 

\xxii)\<sr1/r,  i  =  1, . . . , N,  (1) 

where  T  indexes  the  sorted  coefficients.  Thanks  to  the 
rapid  decay  of  their  coefficients,  such  signals  arc  well- 
approximated  by  Jv -sparse  signals.  Let  xk  £  £a- 
represent  the  best  FT -term  approximation  of  x,  which 
is  obtained  by  keeping  just  the  first  I\  terms  in  xj(i) 
from  (1).  Denote  the  error  of  this  approximation  as 

cfk{x)  :=  arg  min  \\x  —  x\\2  =  \\x  —  .ta'||2- 

We  then  have  that 

&k(x)  <  (rs)-1/2  SK~S,  (2) 


with  s  =  That  is,  the  signal’s  best  approximation 

error  has  a  power-law  decay  with  exponent  s  as  K 
increases.  We  dub  such  a  signal  s-compressible. 

Compressive  sensing  (CS)  integrates  the  signal  acqui¬ 
sition  and  compression  steps  into  a  single  process  [1-3]. 
In  CS  we  do  not  acquire  x  directly  but  rather  acquire 
M  <  N  lineal-  measurements  y  =  <l>x  using  an  M  x  N 
measurement  matrix  <P.  We  then  recover  x  by  exploiting 
its  sparsity  or  compressibility.  Our  goal  is  to  push  M  as 
close  as  possible  to  K  in  order  to  perform  as  much  signal 
“compression”  during  acquisition  as  possible.  In  order  to 
recover  a  good  estimate  of  x  from  the  M  compressive 
measurements,  the  measurement  matrix  <I>  should  satisfy 
the  restricted  isometry  property  (RIP). 

Definition  1:  [1]  An  MxN  matrix  $  has  the  if -RIP 
with  constant  5  k  if,  for  all  x  e  S k, 

(1  -  5k) \\x\\l  <  \\<Z>x\\l  <  (1  +  5k)  \\x\\l-  (3) 


In  words,  the  A  - RIP  ensures  that  all  submatrices  of  $ 
of  size  M  x  K  are  close  to  an  isometry,  and  therefore 
distance  (and  information)  preserving.  Practical  recov¬ 
ery  algorithms  typically  require  that  $  have  a  slightly 
stronger  2  A'-RIP  or  higher-order  RIP  in  order  to  preserve 
differences  of  A'-sparse  vectors  (which  are  2  A'-sparse  in 
general)  and  other  higher-order  structures  [1,6, 12], 

While  the  design  of  a  measurement  matrix  $  sat¬ 
isfying  the  A'-RIP  is  an  NP-Complete  problem  in 
general  [1],  random  matrices  whose  entries  are  i.i.d. 
Gaussian,  Bernoulli  (±1),  or  more  generally  subgaus- 
sian2  work  with  high  probability,  provided  M  = 
O  (I\  log  (IV/ A')).  These  random  matrices  also  have  a 
so-called  universality  property  in  that,  for  any  choice 
of  orthonormal  basis  matrix  T T  has  the  A'-RIP 
with  high  probability.  This  is  useful  when  the  signal 
is  sparse  in  a  basis  other  than  the  identity.  A  random 
$  corresponds  to  an  intriguing  data  acquisition  protocol 
in  which  each  measurement  yj  is  a  randomly  weighted 
lineal-  combination  of  the  entries  of  x. 

A  number  of  different  CS  signal  recovery  algorithms, 
both  from  optimization  and  greedy  approaches  [1-3,5, 
6],  offer  provably  stable  signal  recovery  with  perfor¬ 
mance  close  to  optimal  A' -term  approximation.  For  a 
matrix  $  that  has  the  2A'-RIP  and  noisy  measurements 
y  =  Tx  +  n,  the  recovered  signal  x  holds  the  guarantee 


\x~x\\2  <  C\\\x  -  xk\U  + 

+C3IMI21 


C2 

s/K 


\X  -  XK  1 


(4) 


2  A  random  variable  X  is  called  subgaussian  if  there  exists  c  >  0 
such  that  E  ( eXi )  <  ec2(2^2  for  all  t  £  R.  Examples  include  the 
Gaussian  and  Bernoulli  random  variables,  as  well  as  any  bounded 
random  variable. 
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with  C\  and  C2  denoting  constants.  This  result  has  many 
implications.  Under  noiseless  measurements,  A-sparse 
signals  arc  recovered  perfectly;  under  noisy  measure¬ 
ments,  an  s-compressible  signal  has  recovery  error 


\x  —  x\\2  < 


C,  SK-S  C2SK~s 
+  s -1/2 


+  c^INk- 


(5) 


III.  Model-Sparse  and  Compressible  Signals 

While  many  natural  and  manmade  signals  and  images 
can  be  described  to  first-order  as  sparse  or  compressible, 
the  support  of  their  large  coefficients  often  has  an  under¬ 
lying  inter-dependency  structure.  This  phenomenon  has 
received  only  limited  attention  by  the  CS  community  to 
date  [9-11, 13-16].  In  this  section,  we  present  a  theory 
of  CS  that  captures  such  structure  using  a  union-of- 
subspaces  model.  Such  a  model  reduces  the  degrees  of 
freedom  of  a  sparse/compressible  signal  by  permitting 
only  certain  configurations  of  supports  for  the  large 
coefficients.  As  we  will  show,  this  allows  us  to  reduce, 
in  some  cases  significantly,  the  number  of  compressive 
measurements  M  required  to  stably  recover  a  signal. 

A.  Model-sparse  signals 

Recall  from  Section  II  that  a  A'-sparse  signal  vector 
x  lives  in  E^  C  1^,  which  is  a  union  of  (^.)  subspaces 
of  dimension  A.  Other  than  its  A-sparsity,  there  are 
no  further  constraints  on  the  support  or  values  of  its 
coefficients.  A  union-of-subspaces  signal  model  (a  signal 
model  in  the  sequel  for  brevity)  endows  the  A'-sparse 
signal  x  with  additional  structure  that  allows  certain  A- 
dimensional  subspaces  in  E#  and  disallows  others  [9, 
10].  More  formally,  let  x\<j  represent  the  entries  of  x 
corresponding  to  the  set  of  indices  Q  C  {1 , ,N},  and 
let  dlc  denote  the  complement  of  the  set  0.  A  signal 
model  Mk  is  then  defined  as  the  union  of  mj<  canonical 
A-dimensicnal  subspaces 

m,K 

Mk  =  IJ  Am,  S.t.  Xm  :=  {.T  :  x\nm  e  MA',a;|nc  =  0}; 

m=  1 

each  subspace  Xm  contains  all  signals  x  with  supp(x)  <G 
Qm.  Thus,  the  signal  model  Mk  is  defined  by  the  set 
of  possible  supports  {fli, . . . ,  dlmK}.  Signals  from  M k 
are  called  K-model  sparse.  Clearly,  Mk  C  E k  and 
contains  rriK  <  (^)  subspaces.  In  the  sequel,  we  will 
use  an  algorithm  M(x,  K)  that  returns  the  best  A -term 
approximation  of  the  signal  x  under  the  signal  model 
Mk- 

If  we  know  that  the  signal  x  being  acquired  is  Id- 
mode]  sparse,  then  we  can  relax  the  RIP  constraint  on 
the  CS  measurement  matrix  <b  and  still  achieve  stable 


recovery  from  the  compressive  measurements  y  =  (\>x.  A 
model-based  RIP  requires  that  (3)  holds  only  for  signals 
x  €  Mk  [9, 10];  we  denote  this  new  property  as  M[<- 
RIP  to  specify  the  dependence  on  the  chosen  signal 
model,  and  change  the  model-based  RIP  constant  from 
5k  to  5m  k  for  clarity.  Blumensath  and  Davies  [9]  have 
quantified  the  number  of  measurements  M  necessary 
for  a  subgaussian  CS  matrix  to  have  the  M /c-RIP  with 
constant  5mk  and  with  probability  1  —  e_t  to  be 

M  >  — \ —  ( ln(2mx)  +  A"  In  — +  t )  .  (6) 

C^Mk  k  5mk  ) 

This  bound  can  be  used  to  recover  the  conventional 
CS  result  by  substituting  m-K  =  (x)  ~  (Ne/K)K. 
The  M  a  -RIP  property  is  sufficient  for  robust  recovery 
of  model-sparse  signals  using  recovery  algorithms  like 
model-based  CoSaMP  [8]. 

B.  Model-compressible  signals 

Just  as  compressible  signals  are  “nearly  A'-sparse” 
and  thus  live  close  to  the  union  of  subspaces  E k  in 
M;V,  model-compressible  signals  arc  “nearly  A-modcl 
sparse”  and  live  close  to  the  restricted  union  of  subspaces 
Mk-  To  make  this  new  concept  rigorous,  recall  from 
(2)  that  we  defined  compressible  signals  in  terms  of  the 
decay  of  their  A'- term  approximation  error.  The  I2  error 
incurred  by  approximating  x  €  M;V  by  the  best  model- 
based  approximation  in  Mk  is  given  by 

aMK(x)  :=  _  inf  Ik  -  x\\2  =  ||x  -  M(x,  K)\\2. 
x£Mk 

The  decay  of  this  approximation  error  defines  the  model- 
compressibility  of  a  signal;  we  define  the  set  of  s-model- 
compressible  signals  as 

Wls  =  {xeRN  :  <jMk(x)  <  SK ~l/s, 

1  <  K  <  N,  S  <  00}, 

i.e.,  those  signals  whose  model-based  approximation 
error  has  a  power-law  decay  with  exponent  s  as  K 
increases. 

IV.  The  Restricted  Amplification  Property 
(RAmP) 

In  conventional  CS,  the  same  requirement  (RIP)  is  a 
sufficient  condition  for  the  stable  recovery  of  both  sparse 
and  compressible  signals.  In  model-based  recovery,  how¬ 
ever,  the  class  of  compressible  signals  is  much  larger 
than  that  of  sparse  signals,  since  the  set  of  subspaces 
containing  model-sparse  signals  does  not  span  all  K- 
dimensional  subspaces.  Therefore,  we  need  to  introduce 
some  additional  tools  to  develop  a  sufficient  condition 
for  the  stable  recovery  of  model-compressible  signals. 
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We  will  pay  particular  attention  to  signal  models  Ad  k 
that  generate  nested  approximations,  since  they  are  more 
amenable  to  analysis.  In  words,  a  signal  model  generates 
nested  approximations  if  the  support  of  the  best  K'- 
term  model-based  approximation  contains  the  support 
of  the  best  A'- term  model-based  approximation  for  all 
A  <  K' .  An  important  example  of  a  signal  model 
that  generates  nested  approximations  is  the  standard 
compressible  signal  model  of  (2). 

When  a  signal  model  generates  nested  approximations, 
the  support  of  the  difference  between  the  best  jK- term 
model-based  approximation  and  the  best  (j  +  1)  A-term 
model-based  approximation  of  a  signal  can  be  shown 
to  lie  in  a  small  union  of  A -dimensional  subspaces, 
thanks  to  the  structure  enforced  by  the  signal  model.  This 
structure  is  captured  by  the  set  of  residual  subspaces 
that  are  included  in  each  subsequent  approximation.  We 
group  them  under  the  notation 

=  {u  £  s.t.  for  some  x  £  RN , 
u  =  M(x,  jK)  —  M(x,  (j  —  1)AT)}, 

for  j  =  1, . . . ,  \N/K~\ .  Each  signal  i  in  a  signal 
model  can  be  partitioned  into  its  best  A- term  ap¬ 
proximation  x'Tj ,  the  additional  components  present  in 
the  best  2 A- term  approximation  xt2,  and  so  on,  with 
x  =  xTj  and  XTj  £  Rj,K{M)  for  each  j. 

Each  signal  partition  xt  is  a  A'-sparse  signal,  and  thus 
TZj  k(M)  is  a  union  of  subspaces  of  dimension  A.  We 
will  denote  by  Rj  the  number  of  subspaces  that  compose 
Kj,K(M),  omitting  the  dependence  on  Ad  for  brevity. 

For  exactly  A-model-sparse  signals,  we  discussed  in 
Section  III  that  the  number  of  measurements  M  required 
for  a  random  matrix  to  have  the  AdA'-RIP  is  determined 
by  the  number  of  canonical  subspaces  mx  via  (6). 
Unfortunately,  such  model-sparse  concepts  and  results  do 
not  immediately  extend  to  model-compressible  signals. 
Thus,  we  develop  a  generalization  of  the  Ad  a- RIP  that 
we  will  use  to  quantify  the  stability  of  recovery  for 
model-compressible  signals. 

To  analyze  the  robustness  of  compressible  signal  re¬ 
covery  in  conventional  CS,  we  can  consider  the  tail 
of  the  signal  outside  its  A-term  approximation  as  con¬ 
tributing  additional  “noise”  to  the  measurements  of  size 
||$(x— 2Cat)  || 2  [6, 12].  Consequently,  the  conventional  A'- 
sparse  recovery  performance  result  can  be  applied  with 
the  augmented  noise  n  +  <[>  ( a;  —  x  j< ) .  This  technique 
can  also  be  used  to  quantify  the  robustness  of  model- 
compressible  signal  recovery.  The  key  quantity  we  must 
control  is  the  amplification  of  the  model-based  approxi¬ 
mation  residual  through  <b.  The  following  property  is  a 
new  generalization  of  the  RIP  and  model-based  RIP. 


Definition  2:  A  matrix  $  has  the  (eK,r) -restricted 
amplification  property  (RAmP)  for  the  residual  sub¬ 
spaces  Rj.k  of  the  signal  model  Ad  if 

ll^ull!  <  (i  +  fw)j2rIMl! 


for  any  u  £  Rj.K  and  for  each  1  <j<  \N/K ]. 
Intuitively,  the  norms  of  the  partitions  ||xt.,  ||2  decay 
as  j  increase  for  signals  that  are  compressible  under 
the  signal  model.  This  observation  is  instrumental  in 
relaxing  the  isometry  restrictions  on  the  measurement 
matrix  $  and  bounding  the  recovery  error  for  .s- model - 
compressible  signals.  The  regularity  parameter  r  >  0 
caps  the  growth  rate  of  the  amplification  of  u  £  R,.k 
as  a  function  of  j.  Its  value  can  be  chosen  so  that  the 
growth  in  amplification  with  j  balances  the  decay  of  the 
norm  in  each  residual  subspace  Rj.k  with  j. 

We  can  quantify  the  number  of  rows  M  required  for  a 
random  measurement  matrix  <b  to  have  the  RAmP  with 
high  probability;  we  prove  the  following  in  [8]. 

Theorem  1:  Let  <b  be  an  M  x  N  matrix  with  i.i.d. 
subgaussian  entries  and  let  the  set  of  residual  subspaces 
Tlj,K  of  the  signal  model  Ad  contain  Rj  subspaces  of 
dimension  A  for  each  1  <  j  <  \N/ A].  If 

2A  +  41n^ 

M  >  max  K 


2 1 


1  <j<\N/K]  (jpyi  +  TK  -  1)' 


then  $  has  the  (ex,  r)-RAmP  with  probability  1  —  e_t. 
The  crux  of  the  theorem  is  that  if  the  sequence  { R  j }  has 
slow  growth,  then  a  small  number  of  measurements  will 
suffice  for  robust  recovery  of  model-compressible  sig¬ 
nals.  The  order  of  the  bound  of  Theorem  1  is  lower  than 
O  (K  log  (Ay  A'))  as  long  as  the  number  of  subspaces  Rj 
grows  slower  than  NK .  Armed  with  the  RaMP,  we  can 
state  the  following  result,  which  will  provide  robustness 
for  the  recovery  of  model-compressible  signals;  see  [8] 
for  the  proof. 

Theorem  2:  Let  x  £  9Jis  be  an  .s- model  compressible 
signal  under  a  signal  model  Ad  that  generates  nested 
approximations.  If  $  has  the  (e^-,  r)-RAmP  and  r  = 
s  —  1,  then  we  have 


||<b(x 


M(s,  A))||2  <  y/Y+fiT<SK-s\n 


N 

A 


(7) 


This  result,  combined  with  (4),  provides  us  with  a 
recovery  guarantee  for  compressible  signals: 


with  C\  and  C2  denoting  constants.  For  matrices  having 
the  RAmP,  the  model-based  recovery  algorithms  of  [8] 
provide  guarantees  for  model-compressible  signals  that 
are  similar  to  that  of  standard  algorithms  in  (5). 


x-x\\  <  y-rrz+C2  (  ||n||2  +  v/1  +  crSK  s  In 
K  s 
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V.  Example:  Wavelet  Tree  Model 

Wavelet  decompositions  have  found  wide  application 
in  the  analysis,  processing,  and  compression  of  smooth 
and  piecewise  smooth  signals  because  these  signals  arc 
A'-sparsc  and  compressible,  respectively  [17].  Moreover, 
the  wavelet  coefficients  can  be  naturally  organized  into 
a  tree  structure,  and  for  many  kinds  of  natural  and 
manmade  signals  the  largest  coefficients  cluster  along 
the  branches  of  this  tree.  This  motivates  a  connected 
free  model  for  the  wavelet  coefficients.  We  first  describe 
tree  sparsity  in  the  context  of  sparse  wavelet  decompo¬ 
sitions.  We  focus  on  one-dimensional  signals  and  binary 
wavelet  frees,  but  all  of  our  results  extend  directly  to 
ci-dimensional  signals  and  2'/-ary  wavelet  frees. 

Consider  a  signal  x  of  length  N  =  21 ,  for  an  integer 
value  of  I.  The  wavelet  representation  of  x  is  given  by 

I- 1  24-l 

X  =  V01S  +  EE 

i= 0  j= 0 

where  v  is  the  scaling  function  and  Ay  is  the  wavelet 
function  at  scale  i  and  offset  j.  The  wavelet  transform 
consists  of  the  scaling  coefficient  vq  and  wavelet  coef¬ 
ficients  Wij  at  scale  i,  0  <  i  <  I  —  1,  and  position  j, 
0<j<2i  —  1.  In  terms  of  our  earlier  matrix  notation, 
x  has  the  representation  x  =  fa,  where  \k  is  a  matrix 
containing  the  scaling  and  wavelet  functions  as  columns, 
and  a  =  [ip  (Wo,o  frffio  uti,i  W2,o  ■  ■  ■ }T  is  the  vector 
of  scaling  and  wavelet  coefficients.  We  are,  of  course, 
interested  in  sparse  and  compressible  a. 

The  nested  supports  of  the  wavelets  at  different  scales 
create  a  parent/child  relationship  between  wavelet  coef¬ 
ficients  at  different  scales.  We  say  that  is  the 

parent  of  my  and  that  w.l+ 1  y  and  uti+i  2j+i  are  the  chil¬ 
dren  of  Wij.  Wavelet  functions  act  as  local  discontinuity 
detectors;  using  the  nested  support  property  of  wavelets 
at  different  scales,  it  is  straightforward  to  see  that  a  signal 
discontinuity  will  give  rise  to  a  chain  of  large  wavelet 
coefficients  along  a  branch  of  the  wavelet  tree  from  a 
leaf  to  the  root.  Moreover,  smooth  signal  regions  will 
give  rise  to  regions  of  small  wavelet  coefficients.  This 
“connected  free”  property  has  been  well-exploited  in  a 
number  of  wavelet-based  processing  and  compression 
algorithms  [18, 19].  In  this  section,  we  will  specialize 
the  theory  from  Section  III  to  a  connected  tree  model 
T. 

A  set  of  wavelet  coefficients  (7  forms  a  connected 
subtree  if,  whenever  a  coefficient  wy  €  17,  then  its 
parent  i,|:;/2J  £  17  as  well.  Each  such  set  17  defines 
a  subspace  of  signals  whose  support  is  contained  in  17; 
that  is,  all  wavelet  coefficients  outside  17  are  zero.  In  this 
way,  we  define  the  signal  model  Tk  as  the  union  of  all 


A'-dimensional  subspaces  corresponding  to  supports  17 
that  form  connected  subtrees. 

For  free-compressible  signals,  we  apply  Theorem  1  to 
find  that  a  subgaussian  random  matrix  has  the  ( ck,s )- 
RAmP  for  the  signal  model  T  and  all  s  >  0.5,  with 
probability  1— e_t,  if  the  number  of  measurements  obeys 


20AT  +  4  In  +  2f 
(\/l  +  £K  ~  l) 


(8) 


when  K  >  log2  N ,  or 


M  > 


20 K  +  4  In  2K3+3K2+k  + 
(\/l  +  eK  ~  l) 


(9) 


when  K  <  log2iV;  see  [8]  for  the  proofs.  Both  cases 
give  a  simplified  bound  on  the  number  of  measurements 
required  as  M  =  O(K),  which  is  a  substantial  im¬ 
provement  over  the  M  =  0(Klog(N/K))  required 
by  conventional  CS  recovery  methods.  Thus,  when  <b 
satisfies  (8-9),  we  have  the  guarantee  (7)  for  s-tree- 
compressible  signals. 


VI.  Experiments 

We  now  present  the  results  of  a  number  of  numerical 
experiments  that  illustrate  the  effectiveness  of  model- 
based  recovery.  Our  consistent  observation  is  that  explicit 
incorporation  of  the  tree  model  during  recovery  signifi¬ 
cantly  improves  its  performance  for  a  given  number  of 
measurements. 

We  study  one-dimensional  piecewise  polynomial  sig¬ 
nals  that  match  the  connected  wavelet-tree  model  de¬ 
scribed  above.  We  employ  the  CoSaMP  recovery  al¬ 
gorithm  of  [6]  and  the  model-based  CoSaMP  recovery 
algorithm  of  [8]  using  CSSA,  a  tree-based  approximation 
algorithm  [20],  in  all  experiments. 

Standard  CS  recovery  algorithms  require  that  the 
overmeasuring  factor  M / K  is  approximately  logarithmic 
in  N.  In  contrast,  (8)  suggests  that  stable  CS  recovery 
of  wavelet  tree-compressible  signals  can  be  performed 
with  only  M  =  O  (A)  measurements;  thus,  the  over¬ 
measuring  factor  does  not  increase  with  the  signal  length 
N.  Figure  1  shows  the  growth  of  the  overmeasuring 
factor  with  the  signal  length  N  for  conventional  CS 
and  model-based  recovery.  We  generated  50  sample 
piecewise  cubic  signals  and  numerically  computed  the 
minimum  number  of  measurements  M  required  for  the 
recovery  error  \\x  —  x||2  <  2.5cr-j-K(x),  the  best  tree- 
approximation  error,  for  every  sample  signal.  The  figure 
shows  that  while  doubling  the  signal  length  increases  the 
number  of  measurements  required  by  standard  recovery 
by  K,  the  number  of  measurements  required  by  model- 
based  recovery  is  constant  for  all  N .  These  experimental 
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log2(N) 


Fig.  1.  Required  overmeasuring  factor  M/K  to  achieve  a  target 
recovery  error  \\x  —  a?|| 2  <  2.5otk(x)  as  a  function  of  the  signal 
length  N  for  standard  and  model-based  recovery  of  piecewise  smooth 
signals.  While  standard  recovery  requires  M  to  increase  logarithmi¬ 
cally  with  N,  the  required  M  is  essentially  constant  for  model-based 
recovery. 


results  verify  the  theoretical  performance  described  in 
Section  V. 


VII.  Conclusions 

In  this  paper,  we  have  shown  that  significant  perfor¬ 
mance  gains  can  be  obtained  by  exploiting  more  realistic 
and  richer  signal  models  beyond  the  simplistic  sparse 
and  compressible  models  that  dominate  the  CS  literature. 
Building  on  the  unions  of  subspaces  results  of  [9-1 1],  we 
have  taken  some  of  the  first  steps  towards  what  promises 
to  be  a  general  theory  for  model-based  CS  by  intro¬ 
ducing  the  notion  of  a  model-compressible  signal  and 
the  associated  restricted  amplification  property  (RAmP) 
condition  it  imposes  on  the  measurement  matrix  <I>.  For 
the  volumes  of  natural  and  manmade  signals  and  images 
that  are  wavelet-sparse  or  compressible,  model-based 
CS  offers  performance  that  significantly  exceeds  today’s 
state-of-the-art,  requiring  only  M  =  O  (K)  rather  than 
M  =  0{K\og(N/K))  random  measurements.  Such 
savings  become  significant  for  high-dimensional  signals 
such  as  high  resolution  images. 
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